Tagging Unknown Words with Raw Text Features

Processing unknown words is disproportionately important because of their high information content. It is crucial in domains with specialist vocabularies where relevant training material is scarce, for example: biological text. Unknown word processing often begins with Part of Speech (POS) tagging, where accuracy is typically 10% worse than on known words. We demonstrate that features extracted from large raw text corpora can significantly increase accuracy on unknown words. These features supply a large part of what we are missing with unknown words: context information about how the word is used. We describe a Maximum Entropy modelling approach which uses real-valued features to represent unannotated contextual information. Our initial experiments with real-valued features have resulted in an increased accuracy from 87.39% to 88.85% on unknown words.